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Abstract 



For a given family of spatially coupled codes, we prove that the LP threshold on the BSC of the graph cover 
ensemble is the same as the LP threshold on the BSC of the derived spatially coupled ensemble. This result is in 
contrast with the fact that the BP threshold of the derived spatially coupled ensemble is believed to be larger than 
the BP threshold of the graph cover ensemble HKRU1 II . IKRU12I . To prove this, we establish some properties 
related to the dual witness for LP decoding which was introduced by IF MS+071 and simplified by QDDKW08I . 
More precisely, we prove that the existence of a dual witness which was previously known to be sufficient for 
LP decoding success is also necessary and is equivalent to the existence of certain acyclic hyperflows. We also 
derive a sublinear (in the block length) upper bound on the weight of any edge in such hyperflows, both for regular 
LPDC codes and for spatially coupled codes and we prove that the bound is asymptotically tight for regular LDPC 
codes. Moreover, we show how to trade crossover probability for "LP excess" on all the variable nodes, for any 
binary linear code. 
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1 Introduction 

1.1 Binary linear codes 

A binary linear code £ of block length n is a subspace of the F2-vector space Fg. The e-BSC (Binary Sym- 
metric Channel) with input X £ F% and output Y £ flips each input bit independently with probability 

e. Let 7 be the log-likelihood ratio vector which is given by ji = log ( PY ^ Xi ^\ , A = {—l) Vl log — - for any 

i E {1, . . . ,n}. The optimal decoder is the Maximum Likelihood (ML) decoder which is given by 

/in TT / \ \ U2=iPYi\Xi(yi\xi) 
x M L = argmax pY\x{y\x) = argmax I I py.\ Xi {Vi\Xi) = argmax -= — — 

= argmax log = argmax > log = argmin > 7jXj 



where the second equality follows from the fact that the channel is memoryless. Since the objective function 
is linear in x, replacing £ by the convex span conv{C) of ( does not change the value of the minimal solution. 
Hence, we get 



n 



x M L = argmin ^ 7^ (1) 

xeconv(Q i=1 

ML decoding is known to be NP-hard for general binary linear codes [BMVT78]. This motivates the study 
of suboptimal decoding algorithms that have small running times. 

1.2 Linear programming decoding 

LP (Linear Programming) decoding was introduced by [FWK05 1 and is based on the idea of replacing 
conv(C) in £T|) with a larger subset of R n , with the goal of reducing the running time while maintaining a 
good error correction performance. First, note that conv(C) = conv( f] Q) where Q = {z G {0, l} n : 

w(z\ n(j)) is evenjLj for all j in the set C of check nodes corresponding to a fixed Tanner graph of £ and 
where N(j) is the set of all neighbors of check node j. Then, LP decoding is given by relaxing conv( f] (j) 



'For x £ {0, 1}" and S C {1, . . . , n}, x\s £ {0, 1}™ denotes the restriction of x to S i.e. {x\s)i = Xi if i £ S and {x\s)i 
otherwise, and w(x) denotes the Hamming weight of x. 
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to Pi conv(Cj)'- 

n 

x LP = argmin V%Xi (2) 

where P = f] conv(Cj) is the so-called "fundamental polytope" that will be carefully considered in the 

proof of Theorem 13.21 A central property of P is that it can be described by a linear number of inequalities, 
which means that the linear program d2) can be solved in time polynomial in n using the ellipsoid algorithm 
or interior point methods. 

When analyzing the operation of LP decoding, one can assume that the all-zeros codeword was transmitted 
[FWK05]. Then, by normalizing the expression for the log-likelihood ratio 7 given in Section [Tj] by the 
positive constant log(— -), we can assume that the log-likelihood ratio is given by 7$ = 1 if yi = and 
7i = — 1 if yi = 1 for all i € {1, . . . ,n}. As in previous work, we make the conservative assumption 
that LP decoding fails whenever there are multiple optimal solutions to the linear program ©. In other 
words, under the all zeros assumption, LP decoding succeeds if and only if the zero codeword is the unique 
optimal solution to the linear program (|2). In order to show that LP decoding corrects a constant fraction of 
errors when the Tanner graph has sufficient expansion, llFMS + 07t introduced the concept of a dual witness, 
which is a dual feasible solution with zero cost and with a given set of constraints having a positive slack. 
By complementary slackness, it follows that the existence of a dual witness implies LP decoding success 
[FM S + 07l . A simplified (but equivalent) version of this dual witness, called a hyperflow, was introduced 
in [DDKW08] (and later generalized in [HE12]) and used to prove that LP decoding can correct a larger 
fraction of errors in a probabilistic setting. This hyperflow will be described in Section [3] However, it was 
unkown whether the existence of a hyperflow (or equivalently that of a dual witness) is necessary for LP 
decoding success. We will show, by careful consideration of the fundamental polytope P, that this is indeed 
the case. 



1.3 Spatially coupled codes 

The idea of spatial coupling has been recently used in coding theory, compressive sensing and other fields. 
Spatially coupled codes (or convolutional LDPC codes) were introduced in [JFZ99]. Recently, [K RU11B 
showed that the BP threshold of spatially coupled codes is the same as the MAP (Maximum Aposteriori 
Probability) threshold of the base LDPC code in the case of the Binary Erasure Channel (BEC). Moreover, 
[KRU12] showed that spatially coupled codes achieve capacity under belief propagation. In compressive 
sensing, [ KM S + 12l and BDJM121 showed that spatial coupling can be used to design dense sensing matrices 
that achieve the same peformance as the optimal Zo- norm minimizing compressive sensing decoder. In 
coding theory, the intuition behind the improvement in performance due to spatial coupling is that the check 
nodes located at the boundaries have low degrees which enables the BP algorithm to initially recover the 
transmitted bits at the boundaries. Then, the other transmitted bits are progressively recovered from the 
boundaries to the center of the code. A similar intuition is behind the good performance of spatial coupling 
in compressive sensing [DJ M12II . 



1.4 The conjecture 

It was reported by [Burll | that, based on numerical simulations, spatial coupling does not seem to improve 
the performance of LP decoding. This lead to the conjecture that the LP threshold of a spatially coupled 
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ensemble on the BSC is the same as that of the base ensemble. A natural approach to prove this claim is 
twofold: 

1. Show that the LP threshold of the spatially coupled ensemble on the BSC is the same as that of the graph 
cover ensemble. 

2. Show that the LP threshold of the graph cover ensemble on the BSC is the same as that of the base 
ensemble. 

1.5 Contributions 

We prove the first part of the conjecture. To do so, we prove some general results about LP decoding of 
LDPC codes that may be of independent interest. 

1. We prove that the existence of a dual witness which was previously known to be sufficient for LP decoding 
success is also necessary and is equivalent to the existence of certain acyclic hyperflows (Theorem I3.21 i. 

2. We derive a sublinear (in the block length) upper bound on the weight of any edge in the hyperflow, for 
regular LDPC codes (Theorem 15. Il l and spatially coupled codes (Theorem I6.ll ). In the regular case, we 
show that our bound is asymptotically tight (Theorem 15.1 It . 

3. We show how to trade crossover probability for "LP excess" on all the variable nodes, for any binary linear 
code (Theorem l8.1l >. 

We leave the second part of the conjecture open. 

1.6 Outline 

The paper is organized as follows. In Section |2j we formally state the main result of the paper. In Section [3] 
we prove that the existence of a dual witness which was previously known to be sufficient for LP decoding 
success is also necessary and is equivalent to the existence of certain weighted directed acyclic graphs. In 
Section 01 we show how to transform those weighted directed acyclic graphs into weighted directed forests 
while preserving their central properties. In Section |5] we prove, using the result of Section HJ a sublinear 
(in the block length) upper bound on the weight of any edge in such graphs, for regular codes. An analogous 
upper bound is proved in Section [6] for spatially coupled codes. In Section 13 we relate LP decoding on a 
graph cover code and on a spatially coupled code. In Section [8] we show how to trade crossover probability 
for "LP excess" on all the variable nodes, for any binary linear code. The results of Sections [6] [7] and [8] are 
finally used in Section [9] where we prove the main result of the paper. 

1.7 Notation and terminology 

We denote the set of all non-negative integers by N. For any integers n, a, b with n > 1, we denote by [re] 
the set {1, . . . , re} and by [a : b] the set {a, . . . , b}. For any event A, let A be the complement of A. For 
any vertex v of a graph G, we let N(v) denote the set of all neighbors of v in G. For any x € {0, l} n and 
any S C [n], let x\s £ {0, l} n s.t. (x\s)i = Xi if i G S and (x\s)i = otherwise. A binary linear code £ 
can be fully described as the nullspace of a matrix H € ¥\ , called the parity check matrix of Q. For 

a fixed H, Q can be graphically represented by a Tanner graph (V, C, E) which is a bipartite graph where 
V = {vi, . . . , v n } is the set of variable nodes, C = {c\, . . . , c n -k} is the set of check nodes and for any 
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1 G [n] and any j G [n — k), (vi,Cj) £ E if and only if Hj i = 1. If H is sparse, then Q is called a Low 
Density Parity Check (LDPC) code. LDPC codes were introduced and first analyzed by Gallager [Gal62|. 
If the number of ones in each column of H is d v and the number of ones in each row of H is d c , C is called 
a (d v , d c ) -regular code. We let d v = (d v — l)/2. Throughout the paper, we assume that n, d c , d v > 2. 

2 Main result 

First, we define the spatially coupled codes under consideration. 
Definition 2.1. (Spatially coupled code) 

A (d v ,d c = kd v ,L, M) spatially coupled code, with d v an odd integer and M divisible by k, is constructed 
by considering the index set [-L — d v : L + d v ] and satisfying the following conditions!^ 

1. M variable nodes are placed at each position in [-L : L] and M-^- check nodes are placed at each 
position in [-L — d v : L + d v ]. 

2. For any j 6 [— L + d v : L — d v ], a check node at position j is connected to k variable nodes at position 
j + ifor all i £ [— d v : d v \ 

3. For any j € [-L — d v : — L + d v — 1], a check node at position j is connected to k variable nodes at 
position ifor all i € [-L : j + d v \. 

4. For any j G [L — d v + 1 : L + d v ], a check node at position j is connected to k variable nodes at position 
ifor all i £ [j — d v : L]. 

5. No two check nodes at the same position are connected to the same variable node. 

With the exception of the non-degeneracy condition [5] Definition 12. 1 1 above is the same as that given in 
Section II-A of [KRU11]. We next define the graph cover codes under consideration which are similar to 
the tail-biting LDPC convolutional codes introduced by HTZF07I . 

Definition 2.2. (Graph cover code) 

A (d v ,d c = kd v ,L, M) graph cover code, with d v an odd integer and M divisible by k, is constructed by 
considering the index set [-L : L] and satisfying the following conditions: 

1. M variable nodes and M^- check nodes are placed at each position in [-L : L\. 

2. For any j € [— L : L], a check node at position j is connected to k variable nodes at position (j + i) 
mod [-L : L] for alii € [— d v : d v \ 

3. No two check nodes at the same position are connected to the same variable node. 

Note that "cutting" a graph cover code at any position i G \—L : L] yields a spatially coupled code. This 
motivates the following definition. 

Definition 2.3. (Derived spatially coupled codes) 

Let £ be a (d v , d c = kd v , L, M) graph cover code. For each i G [-L : L], the (d v , d c = kd v , L — d v , M) 
spatially coupled code is obtained from £ by removing all M variable nodes and their adjacent edges at 
each position i + j mod [— L : L]for every j £ [0 : 2d v — 1]. Then, D(C) = {C'-l^ • ■ ■ : ClI i s the set of 
all 2L + 1 derived spatially coupled codes ofQ. 

informally, 2L + 1 is the number of "layers" and M is the number of variable nodes per "layer". 
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Definition 2.4. (Ensembles and Thresholds) 

Let r be an ensemble i.e a probability distribution over codes. The LP threshold £ o/T on the BSC is defined 
as £ = sup{e > | Pr c~r [LP error on £] = o(l)}. 

t-BSC 

We are now ready to state the main result of this paper. 
Theorem 2.5. (Main result: t^cc = Csc) 

Let Tqc be a (d v ,d c = kd v ,L, M) graph cover ensemble with d v an odd integer and M divisible by k. Let 
Tsc be the (d v ,d c = kd v ,L — d v , M) spatially coupled ensemble which is sampled by choosing a graph 
cover code C, ~ ^GC an d returning a element q/"D(£) chosen uniformly at random^ Denote by £gc an d 
^SC the respective LP threholds of Tqc an d ^sc on the BSC. There exists v > depending only on d v and 
d c s.t. if M = o(L v ) and T$c satisfies the property that for any constant A > 0, 

Pr c'~r sc [LP error on ('] = o(—^) (3) 

U SC -A)-BSC L 

Then, £ GC = £sc- 

Note that for M = w(log L), condition (0 above is expected to hold for the spatially coupled ensemble 
Tsc since under typical decoding algorithms, the error probability on the (£sc ~ A)-BSC is expected to 
decay to zero as 0(Le~ cxA xM ) for some constant c > 0. Moreover, note that in the regime M = Q(L S ) 
(for any positive constant 5), spatial coupling provides empirical improvements under iterative decoding and 
in fact, the improvement is expected to take place as long as L is subexponential in M IOU11I . 



3 LP decoding, dual witnesses, hyperflows and WDAGs 

The following definition is based on Definition 1 of IFMS + 07l . 
Definition 3.1. (Dual witness) 

For a given Tanner graph T = (V, C, E) and a (possibly scaled) log-likelihood ratio function 7 : V — > M, 
a dual witness w is a function w : E — » R that satisfies the following 2 properties: 

Vi> E V, ^2 w(v,c) < ^2 (-w(v,c)) +7(«) (4) 

c£N(v):w(v,c)>0 c(=N(v):w(v,c)<0 

Vc G C,Vv,v' G N(c), w(v,c) +w(v',c) > (5) 

The following theorem relates the existence of a dual witness to LP decoding success. The fact that 
the existence of a dual witness implies LP decoding success was shown in IFM S+071 . We prove that the 
converse of this statement is also true. This converse will be used in the proof of Theorem 18. II 

Theorem 3.2. (Existence of a dual witness and LP decoding success) 

Let T = (V, C, E) be a Tanner graph of a binary linear code with block length n and let rj G {0, 1}™ be 
any error pattern. Then, there is LP decoding success for r] onT if and only if there is a dual witness for 77 
on T. 

Proof of Theorem 13.21 See Appendix lA.il □ 

3 Here, 27(C) refers to Definitionl23l 
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The following definition is based on Definition 1 of [DDKW08]. 



Definition 3.3. (Hyperflow) 

For a given Tanner graph T = (V, C, E) and a (possibly scaled) log-likelihood ratio function 7 : V — > HL, 
a hyperflow w is a function w : E — > R that satisfies property above as well as the following property: 

Vc G C,3P C > 0,3v G N(c) s.t. w(v,c) = -P c andW G N(c) s.t. v ^ v, w(v',c) = P c (6) 

By Proposition 1 of |DDKW08), the existence of a hyperflow is equivalent to that of a dual witness. 
Hence, by Theorem 13.21 above, we get: 

Corollary 3.4. (Existence of a hyperflow and LP decoding success) 

Let T = (V, C, E) be a Tanner graph of a binary linear code with block length n and let r/ G {0, 1}™ be any 
error pattern. Then, there is LP decoding success for r\onT if and only if there is a hyperflow for ri on T. 

Definition 3.5. (WDG corresponding to a hyperflow or a dual witness) 

Let T = ( V, C, E) be a Tanner graph, 7 : V — > R a (possibly scaled) log-likelihood ratio function and 
w : E — > Ma dual witness or a hyperflow. The weighted directed graph (WDG) (V, C, E, w, 7) associated 
with T, 7 and w has vertex setVUC and for any v G V and any c G C, an arrow is directed from v to cif 
w(v, c) > 0, an arrow is directed from c to v ifw(v, c) < and v and c are not connected by an arrow if 
w(v, c) = 0. Moreover, a directed edge between v (zV and c G C has weight \ w(v, c)|. 

The following theorem shows that whenever there exists a WDG corresponding to a hyperflow or a dual 
witness, there exists an acyclic WDG (denoted by WDAG) corresponding to a hyperflow. 

Theorem 3.6. (Existence of an acyclic WDG) 

Let T = (V, C, E) be a Tanner graph of a binary linear code with block length n and let rj G {0, 1}™ be any 
error pattern. IfG = (V,C,E,w,~f) is a WDG (Weighted Directed Graph) corresponding to a dual witness 
for rj on T, then there is an acyclic WDG G" = (V, C, E, w" , 7) corresponding to a hyperflow for rj on T. 

Before proving Theorem 13.61 we summarize the different characterizations of LP decoding success. 

Theorem 3.7. Let T = (V, C, E) be a Tanner graph of a binary linear code with block length n and let 
rj G {0, 1}™ be any error pattern. Then, the following are equivalent: 

1. There is LP decoding success for rj on T. 

2. There is a dual witness for rj on T. 

3. There is a hyperflow for rj on T. 

4. There is a WDAG for ry on T. 

In order to prove Theorem 13.61 we give an algorithm that transforms a WDG G satisfying Equations (01) 
and © into an acyclic WDG G" satisfying Equations (@]) and ((6]). 



7 



Input: G = (V, C, E, w, 7) 
Output: G" = (V, C, E, w",j) 



G' = (V,C,E,w',~/) <- G 
while G' has a directed cycle do 
c <— any directed cycle of G' 

w m i n <— minimum weight of an edge of c > All edges along c have a positive weight. 

Subtract w m i n from the weights of all edges of c 
Remove all zero weight edges 
Store the resulting WDG in G' 
end while 



for all j G C do 

d(j) <— degree of j 

{vi, . . . , iWj)} <— neighbours of j in order of increasing w'(vi,j) 

if w'{v\ , j) > then > All edges are directed toward j and can thus be removed. 

w"{v h j)^Wie[d{j)\ 
else > (v\ ,j) is the only edge directed away from j. 

w"(v u j) <- w'{vi,j) 

w"(v h j)^\w'(v u j)\Vi£{2,...,d(j)} 
end if 
end for 

Algorithm 3.1: Transforming the dual witness WDG G for 7 into a hyperflow WD AG G" for 7 



The next lemma is used to complete the proof of Theorem 13.61 

Lemma 3.8. After each iteration of the while loop of Alsorithm \3. 1\ we have: 

(I) The number of cycles of G' decreases by at least 1. 

(II) G' satsifies the dual witness equations (0) and (0. 

Proof of Lemma [3l8l (I) follows from the fact that cycle c is being broken in every iteration of the while 
loop and no new cycle is added by reducing the absolute weights of some edges of the WDG. (II) follows 
from the fact that during any iteration of the while loop, we are possibly repeatedly reducing the absolute 
weights of one ingoing and one outgoing edge of a variable or check node by the same amount, which 
maintains the original LP constraints dD and ©. □ 

Proof of Theorem 13.61 First, note that the while loop of Algorithm 13. II will be executed a number of times 
no larger than the number of cycles of G, which is finite. By Lemma [3T8l after the last iteration of the while 
loop, G' is an acyclic WDG that satisfies (0]) and (f5]). The for loop of Algorithm 13 . 1 I decreases the weights of 
edges that are directed away from variable nodes; thus, it maintains (0]) and G" inherits the acyclic property 
of G' . Moreover, G" satsifies ©, which completes the proof Theorem 13.61 □ 

Remark 3.9. In virtue ofTheorem \3.2\ Theorem \3.4\ and Theorem \3.6\ we will use the terms "hyperflow", 
"dual witness" and "WDAG" interchangeably in the rest of this paper. 
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4 Transforming a WDAG into a directed weighted forest 



The WDAG corresponding to a hyperflow has no directed cycles but it possibly has cycles when viewed as 
an undirected graph. In this section, we show how to transform the WDAG corresponding to a hyperflow 
into a directed weighted forest (which is by definition a directed graph that is acyclic even when viewed as 
an undirected graph). This forest has possibly a larger number of variable and check nodes than the original 
WDAG but it still satisfies Equations (@]) and (©. Moreover, the vertices of the forest "corresponding" 
to a vertex of the original WDAG will have their weights sum up to the weight of the original vertex. 
Furthermore, the directed paths of the forest will be in a bijective correspondence with the directed paths of 
the original WDAG. This transformation will be used when we derive an upper bound on the weight of an 
edge in a WDAG of a (d v , d c ) -regular LDPC code in Section[5]and of a spatially coupled code in Section|6] 

Theorem 4.1. (Transforming a WDAG into a directed weighted forest) 

Let G = (V, C, E, w, 7) be a WDAG. Then, G can be transformed into a directed weighted forest T = 
(V, C', E', w' , 7') that has the following properties: 

1. V = (J Vy where n Vy = %for all x,y £ V s.t. x 7^ y. For every v G V, each variable node in is 

vev 

called a "replicate" of v. 

2. C' = |J C' c where C' x Pi C' = %for all x,y € C s.t. x ^ y. For every c S C, each check node in C' c is 

cec 

called a "replicate" of c. 



4. For all v and all v' £ V v , 7'(V) has the same sign as 7(f). 

5. The forest T satisfies the hyperflow equations (0) and ((6|). 

6. The directed paths of G are in a bijective correspondence with the directed paths of T. Moreover, if the 
directed path h' ofT corresponds to the directed path h of G, then the variable and check nodes of hi are 
replicates of the corresponding variable and check nodes ofh. 

7.IfG has a single sink node with a single incoming edge that has weight a, then T has a single sink node 
with a single incoming edge and that has the same weight a. 

In order to prove Theorem 14. II we now give an algorithm that transforms the WDAG G into the directed 
weighted forest T. 
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Input: G = (V, C, E, w, 7) 
Output: T = (V, C, E' , w',Y) 

for each v G V taken in topological order do 
p <— number of outgoing edges of v 
{ j }^=i weights of outgoing edges of v 

Create p replicates of the subtree rooted at v > Contains all ancestors of v in the current WDAG 
for each I G \p) do 

Scale the Zth subtree by e; / ej) > The weights of all variable nodes and edges are scaled 

Connect the Zth subtree to the Zth outgoing edge of v 
end for 
end for 

Algorithm 4.1: Transforming the WDAG G into the directed weighted forest T 

We now state and prove a loop invariant that constitutes the main part of the proof of Theorem 14. 11 First, 
we introduce some notation related to the operation of Algorithm 14.11 

Notation 4.2. In the following, let V = {v±, . . . ,v n }. For every i,j G [n], let rij be the number of 
replicates of variable node vj after the ith iteration of the algorithm. Moreover, for every k G let Vij^k 
be the kth replicate of Vj after the ith iteration of the algorithm. For all i € [n], let Vi, C{, £7j, 7, and Wi be 
the set of all variable nodes, set of all check nodes, set of all edges, log-likelihood ratio function and weight 
function, respectively, after the ith iteration of the algorithm and let Gi = (Vi,Ci, Ei,Wi,~/i). Finally, we 
set G = (y ,Co,E ,jo,w Q ) to (V, C, E, 7, w). 

Lemma 4.3. For any i > 0, after the ith iteration of Algorithm \4.1\ we have^ 

r i,j 

(I) For all j G [n], ^7i(^,j,fc) = 
k=l 

(II) For all j G [n] and all k G [rij], n fi{vi j k) has the same sign as j(vj). 

(III) For all v e Vi, ^ Wi(y,c) < ^ (-Wi(y, c)) + %{v). 

c£N(v):Wi(v,c)>0 c£N(v):Wi(v,c)<0 

(IV) For all c G Cj, there exist P c > and v G N(c) s.t. Wi(v,c) = —P c and for all v' G N(c) s.t. 
v' ^ v,Wi(v',c) = P c . 

(V) The directed paths of G are in a bijective correspondence with the directed paths of 'Gi. Moreover, if the 
directed path h! of Gi corresponds to the directed path h of G, then the variable and check nodes of hi 
are replicates of the corresponding variable and check nodes of h. 

Proof of Lemma l4.31 Base Case: Before the first iteration, we have: ro,j = 1 , 7o(^0,i,i) = li v j) f° r ai l 
j G [n]. Thus, (I) and (II) are initially true. (Ill) and (IV) are initially true because the original WDAG G 

4 By "after the 0th iteration", we mean "before the 1st iteration". 
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satisfies the hyperflow equations © and ©. Moreover, (V) is initially true since Go — G. 

Inductive Step: We show that, for every i > 1, if (I), (III), (IV) and (V) are true after iteration i — 1 of 

Algorithm 14.11 then they are also true after iteration i. 

Let % > 1. In iteration i, a variable node v with log-likelihood ratio 7j_i(f) is (possibly) replaced by a 
number p of replicates {v[ , . . . , v' p } with log-likelihood ratios { -^y7i_i (f) / G [p]}. Therefore, the total 



sum of the added replicates is 




and since e\ / > 0, it follows that (II) is also true. 

To show that (III) is true, we first note that if v' G Vi was not created during the zth iteration, then v' will 
satisfy (III) after the ith iteration. If v' was created during the zth iteration, we distinguish two cases: 
In the first case, v' is not a replicate of v (which is the variable node considered in the ith iteration). Then, 
v' is a replicate of Vi-i G Vi-\. By the induction assumption, 7j_i(f j-i) and the weights of the adjacent 
edges to V{-i satisfy (III) before the ith iteration. Since ji(v') and the weights of the edges adjacent to v' 
will be respectively equal to 7i-i(t>j-i) and the weights of the edges adjacent to scaled by the same 
positive factor, v' will satisfy (III) after the ith iteration. 

In the second case, v' is a replicate of v. Assume that v' is the replicate of v corresponding to the edge 
(v,cq) where cq G N(v) and Wi-i(v, cq) > 0. During the ith iteration, the subtree corresponding to v' 
will be created and in this subtree, 7i(u') and the weights of the edges incoming to v' will be respectively 
equal to 7j_i(-u) and the weights of the edges incoming to v, scaled by 6(v, cq) = Wi_i{v, co)/e^ where 
e T = 5^ w i-i( v i c )- The only outgoing edge of v' will be (v', Co). Thus, 

Wi-i(v, c ) = 9(v, c ) ^2 Wi-i(y,c) 

ceN(v)-.Wi-i(v,c)>0 

<0(v,co)( ^2 (-Wi-i(v,c)) +7»_i(«) 

c£N(v):Wi-i(v,c)<0 

= 8(v,c ) ^2 (-Wi-i{v,c)) +6(v,co)ji 

c£N(v):Wi-i(v,c)<0 

Yl (-«'*(«', c))+7 i (u / ) 

c£N(v'):Wi(v',c)<0 

Therefore, v' will satisfy (III) after the ith iteration. 

Equation (IV) follows from the induction assumption and from the fact that we are either uniformly scaling 
the neighborhood of a check node or leaving it unchanged. 

To prove that (V) is true after the ith iteration, let v be the variable node under consideration in the ith 
iteration and consider the function that maps the directed path h of Gi—\ to the directed path h! of G{ as 
follows: 

1. If h does not contain v, then h! is set to h. 

2. If h contains v, then h can be uniquely decomposed into the concatenation h\h,2 where h\ is a directed 
path of Gi-i that ends at v and hi is a directed path of Gj_i that starts at v. Let ei be the first edge of h%. 
Then, h! is set to b!-Ji<2 where b! x is the directed path in the Zth created subtree of G' that corresponds to 



22 Wi(v', c) = U>i(v', c ) = 

ceJV(t)'):tUi(«',c)>0 
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This map is a bijection from the set of all directed paths of Crj_i to the set of all directed paths of Gj. 
Moreover, if the directed path h of Gj_i is mapped to the directed path h' of Gi, then the variable and check 
nodes of h' are replicates of the corresponding variable and check nodes of h. 

□ 

Proof of Theorem 14. 11 Note that Q] and |2] in Theorem 14.11 follow from the operation of Algorithm 14.11 
Moreover, mm |5] and |6] follow from Lemma 1431 with 7' = 7„. To prove |7J note that if G has a single sink 
node v, then v will be the last vertex in any topological ordering of the vertices of G. Furthermore, if v has 
a single incoming edge with weight a, then it will have only one replicate in T, with a single incoming edge 
having the same weight a. 

□ 

5 Maximum weight of an edge in a regular WDAG on the BSC 

In this section, we present sublinear (in the block length n) upper bound on the weight of an edge in a 
regular WDAG. The main idea of the proof is the following. Consider a (d v , d c )-regular WDAG G (where 
d v ,d c > 2 we, constants) corresponding to a hyperflow. Note that each variable node has a log-likelihood 
ratio of ±1. Thus, the total amount of flow available in the WDAG is most n. Moreover, for a substantial 
weight to get "concentrated" on an edge in the WDAG, the +l's should "move" from variable nodes accross 
the WDAG toward that edge. By the hyperflow equation ©, each check node cuts its incoming flow by 
a factor of d c — 1. Thus, it can be seen that the maximum weight that can get concentrated on an edge is 
asymptotically smaller than n. 

Theorem 5.1. (Maximum weight of an edge in a regular WDAG on the BSC) 

Let G = (V, C, E, w, 7) be a WDAG corresponding to LP decoding of a (d v ,d c ) -regular LDPC code (with 
d v , d c > 2) on the BSC. Let n = \V\ and a max = max|w;(e)| be the maximum weight of an edge in G. 

Then, 

ln(d„-l) 

Oimax < Cn ln <d«-l)+ln(d c -i) = ( n ) (7) 

for some constant c > depending only on d v . 

We now state and prove a series of lemmas that leads to the proof of Theorem 15. II 
Definition 5.2. (Root-oriented tree) 

A root-oriented tree is defined in the same way as the WDAG in Definition \3. 3\ and Theorem \3.6\ but with the 
further constraints that T has a single sink node (which is a variable node) and that T is a tree when viewed 
as an undirected graph. Note that the name "root-oriented" is due to the fact that the edges are oriented 
toward the root of the tree, as shown in Figure\l\ 

Remark 5.3. Algorithm \4. 1 1 can also be used to generate the directed weighted forest corresponding to the 
subset of the WDAG consisting of all variable and check nodes that are ancestors of a given variable node 
v. In this case, the output is a root-oriented tree with its single sink node being the unique replicate of v. 

Definition 5.4. ( G max , a max ) 

Let G = (V, C, E, w, 7) be a WDAG. Let e max = (v max ,c max ) = argmax \w(v, c)| and let a max = 

(v,c):w(v,c)<0 

\w(v max , c max )\. Let V max = V\ U {v max } where V\ is the set of all variable nodes v G V s.t. c max is 
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Figure 1 : Root-oriented tree with root the variable node vq 



reachable from v in G and let Crnax the set of all check nodes c G C s.t. c max is reachable from c in 
Let G max = (V m ax,C max ,E max ,w max ,'y max ) be the corresponding WDAG. 

Definition 5.5. (Depth of a variable node in a root-oriented tree) 

Let T be a root-oriented tree with root vq. For any variable node v in T, the depth ofv in T is defined to be 
the number of check nodes on the unique directed path from v to vq in T. 

Definition 5.6. ( F -function ) 

Let G = {V, C, E, w, 7) be a WDAG. For any S C V, define F(S) = ^ w(v, c). In other 

veS c£N(v):w(v,c)>0 

words, F(S) is the sum of all the "flow" leaving variable nodes in S to adjacent check nodes. 

Lemma 5.7. Let G = (V,C,E,w,j) be a WDAG corresponding to LP decoding of a (d v ,d c ) -regular 
LDPC code (with d v ,d c > 2) on the BSC and let G max = (V max , C max , E max , w max , ^ max ) be the WDAG 
corresponding to Definition \5.4\ Let n max = \V max \ and T = (V', C',E', w' , 7') be the output of Algorithm 
\4.1\ on input G max . Note that T is a root-oriented tree with root v max which has a single incoming edge 
with weight a max (by Theorem \4.1\) . Let d max be the maximum depth of a variable node in T and for any 
m £ {0, . . . , d max }, let S m be the set of all variable nodes in T with depth equal to m. Moreover, for all 
i € {0, . . . , d max } and all j € [n max ], let dij denote the number of replicates of variable node Vj having 
depth equal to i in T. Furthermore, for every k € [dij], let Tijk be the 7' value of the kth replicate of Vj 
among those having depth equal to i in T. Then, for all m £ {1, . . . , d m ax}, we have: 

TO— 1 nmax 

(P m ) : F(S m ) > (d c - l) m a max - ( d c ~ 1)™"' £ T ^ < 8) 

i=0 j=l k=l 

Proof of Lemma l5Jl For any S C V', let A(S) be the set of all v G V for which there exist s G S and a 
directed path from v to s in T containing exactly one check node. We proceed by induction on m. 
Base Case: m = 1. We note that S\ = A({v max }) and that v max is the only variable node in T having 
depth equal to in T. Hence, for the hyperflow to satisfy ©, we should have: 

Hmoi "»J 

F(Si) > (d c - l)(a max - j'(v max )) = (d c - l)a max - - l) 1 ^ ^ r i,j,k 

i=0 j=l k=l 

Note that the last equality follows from the facts that do,j = 1 if Vj = v max and doj = otherwise, and that 

Fi,j,k = iiymax) if Vj = v max and k = 1 and T i>j>k = otherwise. 

Inductive Step: We need to show that if (Pm) is true for some 1 < m < d max — 1, then (P m+ \) is also true. 



- Note that c max € C„ 
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Assuming that (P m ) is true, S m satisfies Equation ([8]). Since T is a root-oriented tree, S m+ \ = A(S m ). 
Hence, for the hyperflow to satisfy ©, we should have: 

F(S m+1 ) > (dc - l)(F(S m ) EE Tm ^ k ) 

3=1 k=l 

m—l n max TLmax d m j 

> {dc _ i)[( dc _ l)^a max - J^idc - l) m - 1 E E r *J.* EE T rnj,k] 

1=0 3=1 fc=l 3=1 fe=l 

Wt Umax 

= (d c - l) m+1 a max - J2(d c - l) m+1 ~* £ E r *J> 

i=0 3=1 fe=l 

□ 

Definition 5.8. (Depth of a variable node in a WDAG with a single sink node) 

Let G = (y,C,E,w,'y) be a WDAG with a single sink node v o G V and let v G V. The depth of v in G is 
defined to be the minimal number of check nodes on a directed path from v to vq in G. 

Corollary 5.9. Let g ma x be the maximum depth of a variable node v G V max in the WDAG G max ( which 
has a single sink node v max Then, 

ctmax < max /(T , ... , T 9max ) (9) 

(l ,...,lg max )€W 



where: 

/(T , . . . ,T gmax ) = 



<l:n,i.r 



i=0 

and W is the set of all tuples (To, . . . , T gmax ) G N 9ma:i:+1 satisfying the following three equations: 

Qmax 

E Ti = Umax ^ 

i=0 

2b = 1 (11) 
For all i G {0, . . . , g m ax ~ 

l},T i+1 <(d c -l)(d v -l)Ti (12) 

Proof of Corollary 15.91 Setting m = d max in Lemma 15771 and noting that the leaves of T have no entering 
flow, we get: 

Umax d-maxij dmax 1 T^max 

E E Td max ,3,k>F(s dmax )>(d c -i) d -*a max - e a-i^-EE^ 

j=l k=l i=0 3=1 fe=l 

Thus, 



"max < E M _ 1 y E E ^ i >j> k 
i=0 ^ c > 3=1 k=l 



6 Note that in general g max < dmax but the two quantities need not be equal. 



14 



Part [6] of Theorem 14. 1 1 implies that for all v G V max , the depth of v in G max is equal to the minimum 
depth in T of a replicate of v. By parts [3] and 0] of Theorem 14.11 we also have that for all j G [n maa; ], 

dmax 

^ ^ r * - 1 and for al1 i e {°> • • • ' d max} and all k G [dij], r iij>k < 1 and {Tij jk } ijk all have 

the same sign. For every j G [n max ], let <ij be the depth of vj in G max and note that dj < i for every 
% G {0, . . . , dmax} for which there exists A; G [djj] s.t. T^ft 7^ 0. Thus, we get that: 



^max ' <"max ' <>max -y '-"max "jj '-"max 

°"- - § k^tf E g i r «.*i s £ ^^n^ g g ir«»i = g j^t. 



dmax dmax 

where the last equality follows from the fact that |rjj,fc| = I XT = ^ ^ or ever y 3 e 

i=0 fe=l i=0 fc=l 

[n mai ] with Tj being the number of variable nodes with depth equal to i in G max for every i G [d max \. Note 

that the notion of depth used here is the one given in Definition 15.81 since G max is a WD AG with a single 
sink node v max . Since T t = for all g ma:E < % < d max , we get: 



Qmax 

\d c - iy 



i=0 

Equations (fTOl) . (fTTT) and (PT2l ) follow from the definitions of Tj and g maa; . □ 

ln(d„-l) 

Lemma 5.10. The RHS of Equation d9]) is at most c x {n max ) in(d«-i)+in(d c -i) y r some constant c > 
depending only on d v . 

Proof of Lemma fe-lPl Follows from Theorem I A. 6 1 with A = 1, /3 = (d c — l)(d v — 1) and m = n max . □ 

Proof of Theorem [57TJ Theorem 15.11 follows from Corollary |5.9| and Lemma |5. 10 I by noting that |y max | < 

|V| since V max C V and that max |u>(e)| = 0( max c)|) by the hyperflow equation ©. □ 

eg_E (u,c):w(u,c)<0 

We now show that the bound given in Theorem 15 .H is asymptotically tight in the case of (d v , d c )-regular 
LDPC codes. 

Theorem 5.11. (Asymptotic tightness of Theorem 15. 1 \ for (d v , d c )-regular LDPC codes) 
There exists an infinite family of (d v , d c )-regular Tanner graphs {(V n , C n , E n )} n , an infinite family of error 
patterns {j n }n and a positive constant c s.t. there exists a hyperflow for j n on (V n , C n ,E n ) and any WDAG 
(V n , C n , E n , w, 7„) corresponding to a hyperflow for 7 n on (V n , C n ,E n ) must have 

max|w(e)| > cn ln ( d "- 1 )+ ln ( d c- 1 ) 

Proof of Theorem See Appendix |A3J □ 
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6 Maximum weight of an edge in the WDAG of a spatially coupled code on 
the BSC 



The upper bound of Theorem 15 . 1 I holds for (d v , d c ) -regular LDPC codes. In this section, we derive a similar 
sublinear (in the block length n) upper bound that holds for spatially coupled codes. 

Theorem 6.1. (Maximum weight of an edge in a spatially coupled code) 

Let G = (V, C, E, u>, 7) be a WDAG corresponding to LP decoding of any code of the (d v , d c = kd v , L, M) 
spatially coupled ensemble on the BSC. Let n = (2L + 1)M = \V\ be the block length of the code. Let 
otmax = max 1 10(e) I be the maximum weight of an edge in G. Then, 

e£E 

ln(q)-ln(d c -l) 

otmax < cn M?) = cn 1 e = (n) (13) 

for some constant c > depending only on d v and where q = d v (d c — 1) ^ dv ~ d 1 ^_ 2 ~* an d < e = ^j^^ < 
1. 

We now state and prove a series of lemmas that leads to the proof of Theorem 16.11 Note that a central 
idea in the proof of Section [5] is that all check nodes being d c -regular in that case, the flow at every check 
node is "cut" by a factor of d c — 1. On the other hand, a (d v = 3, d c = 6, L, M) spatially coupled code 
has 2M check nodes with degree 2 and the flow is preserved at such check nodes. To show that even in this 
case, the maximum weight of an edge is sublinear in the block length, we argue that a check node that is not 
d c -regular should have a d c -regular check node that is "close by" in the WDAG. To simplify the argument, 
we first "clean" the WDAG of the spatially coupled code to obtain a "reduced WDAG" with all check nodes 
having either degree d c or degree 2. We also use a notion of "regular check depth" which is the same as the 
notion of depth of Section [6j] except that only d c -regular check nodes are now counted. 

Definition 6.2. (Reduced WDAG) 

Let G = (V, C, E, w, 7) be a WDAG and G max = (V max , C max , E max , w max , j max ) be the WDAG corre- 
sponding to Definition 15.41 The reduced WDAG G r of G max is obtained by processing G max as follows so 
that each check node has either degree d c or degree 2: 

1. For every check node c of G r with spatial inde$\ < (— L + d v ), we remove all the incoming edges to c 
except one that comes from a parent of c having maximal spatial index. 

2. For every check node c ofT' with spatial index > (L — d v ), we remove all the incoming edges to c except 
for one edge that comes from a parent of c having minimal spatial index. 

3. We keep only the variable nodes v s.t. v max is still reachable from v and the check nodes c s.t. v max is 
still reachable from c. 

Note that in steps\l\and\2\above, the check nodes of G r are considered in an arbitrary order. 
Definition 6.3. (Reduced tree) 

A reduced tree with root v$ is a root-oriented tree with root vq and where every check node has either degree 
d c or degree 2. 

7 The notion of "spatial index" used here is the one from Definition ^. II 

8 The notion of "parent" of a node is the one induced by the direction of the edges of G r . 
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Note that if we run Algorithm 14. li on a reduced WD AG, the output will be a reduced tree. 



Definition 6.4. (Regular check depth of a variable node in a reduced tree) 

Let T be a reduced tree with root vq. For any variable node v ofT, the regular check depth of v in T is the 
number of d c -regular check nodes on the directed path from v to vq in T. 

Lemma 6.5. Let G = (V, C, E, w,'y) be a WDAG corresponding to LP decoding of a spatially coupled 
code on the BSC, G max = (V max ,C max ,E max ,w max ,j max ) be the WDAG corresponding to Definition 
15.41 G T = (V r , C r , E r , w r ,j r ) be the reduced WDAG corresponding to G max and T = (V/, C' r ,E' r , w' r , j' r ) 
be the output of A leorithm \4. 1 1 on input G r . Let n r = \V r \. Note that T is a reduced tree with root v max 
which has a single incoming edge with weight a max (by Theorem \4. 1 I ). Let r max be the maximum regular 
check depth in T of a variable node v € V^. For all i £ {0, . . . , r max } and all j £ [n r ], let y^j be the num- 
ber of replicates of variable node Vj having regular check depth equal to i in T. Moreover, for all k E [yij], 
let Tij k denote the j' r value of the kth replicate ofvj among those having regular check depth equal to i in 
T. Then, for all m 6 {1, . . . , r max }, we have: 

(P m ): There exists U m C V£ consisting of variable nodes having regular check depth m in T and s.t. 
all variable nodes ofT having regular check depth between m + 1 and r max (inclusive) are ancestors of 
U m in T and s. t. : 

m—l n r Vi,j 

F(u m ) > (d c - i) m a max - J2( d c - ir _i EE r ^ < 14 > 

i=0 j=l k=l 

Proof of Lemma 1631 For any S C V^, let A(S) be the set of all v G for which there exist s G S and a 
directed path from v to s in T with the child of v on this path being the unique d c -regular check node on the 
pathJl We proceed by induction on m. 

Base Case: m = 1. Let U\ = A({v max }). Note that the ancestors of v max (inlcuding v max ) that are proper 
descendants of nodes in U\ are exactly those variable nodes having regular check depth equal to in T. 
Hence, for the hyperflow to satisfy Equation ©, we should have: 

n r Vo, j n r Vi,i 

F{U\) > (d c - l)(a max - ^ ^ r o,j,fc) = (d c - l^awc - ^(d c - I) 1 X!Xl r M. fc 

j=l k=l i=0 j=l k=l 

Inductive Step: We need to show that if (P m ) is true for some 1 < m < (r max — 1) then (P m+ i) is also 
true. Assuming that (P m ) is true, there exists U m C V£ that satisfies Equation (fT4l and s.t. U m consists 
of variable nodes having regular check depth m in T, and all variable nodes of T with regular check depth 
between m + 1 and r max (inclusive) are ancestors of U m in T. Let U m+ i = A(U m ). Note that the variable 
nodes that are ancestors of nodes in U m and proper descendants of nodes in U m+ \ are exactly those having 



9 Again, the notion of "child" here is the one induced by the direction of the edges of T. 
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regular check depth equal to m in T. Hence, for the hyperflow to satisfy Equation ©, we should have: 

F(U m+1 ) > (d c - l){F{U m ) 

j=i k=i 



> (4- 


m— 1 

l)[(d c -l) m a max - J2( d c 


n r Vi,j 


i,k ~ Yl Yl ^ m J' fc ] 




i=0 


3=1 k=l 


j=l k=l 




m—1 




n r VrnJ 


= (4- 


l) m+1 ama X -^2(dc-l) m - 




'(^ _1 )EE r rn,j,k 




i=0 


j=i k=i 


j=l k=l 


= (d c - 


m 

l) m+1 cw - ^(4 - l) m4 
i=0 


n r Vi,j 

Yl Yl Ti >j> k 
j=i k=i 





□ 

Definition 6.6. (Regular check depth of a variable node in a reduced WDAG) 

Let G r be a reduced WDAG with its single sink node denoted by vq. For any variable node v of G r , the 
regular check depth of v in G r is the minimum number of d c -regular check nodes on a directed path from v 
to vq in G r . 

Lemma 6.7. Let G r be a reduced WDAG and z max be the maximum regular check depth of a variable node 
in G r . For all i € {0, . . . , z max }, let Ti be the number of variable nodes in G r with regular check depth 
equal to i. Then, for all i £ {0, . . . , z max — 1}: 

Ti+i < qTi 

where q = d v (d c — 1) ^ t, ^ 1 I 2 ~ - Moreover, Tq < 1 + ^"~^_ 2 ~ 1 = Qo- 

Proof of Lemma RT7l If, for any i G {0, . . . , z max }, we let Wi be the set of all variable nodes in G r with 
regular check depth equal to i, then Tj = |Wj|. Fix i S {0, . . . , z max — 1}. For a variable node v of G r , 
define A'(v) to be the set of all variable nodes vo in G r s.t. there exists a directed path V from vq to v 
in G T s.t. the parent of v on V is the only d c -regular check node on V. Note that for every variable node 

u G Wt+i» th ere exists a variable node v G Wi s.t. u G A'(v). Thus, Wj+i C (J A'(v) which implies that 

veWi 

\W i+ i\ < \Wi\ x max|A'(u)| < \WA x max|A'(u)| 

v£Wi vev r 

where V r is the set of all variable nodes of G r . We now show that for every v G V r , \A'(v)\ < q. Fix v G V r . 
We claim that for all u G A'(v), there exists a directed path from u to v in G r containing a single d c -regular 
check node which is the parent of v on this path and at most (d v — 1) 2-regular check nodes. To show this, 
let V be a directed path from u to v in G r containing no d c -regular check nodes other than the parent of v 
on this path. If V does not contain any 2-regular check nodes, then the needed property holds. If V contains 
at least one 2-regular check node, then, 

'P I U ~^ C% V\ C2 V2 • • • Q Vl <w c* ~^ v (15) 

where I is a positive integer, c\, C2, • • • , Q are 2-regular check nodes of G r , is a d c -regular check node of 
G r and «i, V2, ■ ■ ■ ,vi are variable nodes of G r . For any check node c, we denote by si(c) the spatial index 
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of c. Since c\ is 2-regular, its spatial index si{c\) is either in the interval [— L — d v : —L + d v — 1] or in the 
interval [L — d v + 1 : L + d v ]. Without loss of generality, assume that si(c\) G [L — d v + 1 : L + d v ]. For 
any i G {0, ...,/ — 1}, Definition 16.21 implies that i>j is at a minimal position w.r.t. c, + i. By Definition ^. II 
if variable node v is at a minimal position w.r.t. check node c, then c is at a maximal position w.r.t. v. So for 
any i G {0, . . . , I — 1}, q+i is at a maximal position w.r.t i>j and thus si(ci) < si(ci + \). By condition [5] of 
Definition ^. II variable node is not connected to two check nodes at the same position, which implies that 
si(ci) 7^ si(ci + i) for alH G {0, . . . , I — 1}. So we conclude that si(ci) < si(cj+i) for all i G {0, . . . , I— 1}. 
Therefore, 

L — d v + 1 < si(c\) < si(c2) < ■ ■ ■ < si{pi) < L + d v 
Hence, / < 2d v = d v — 1. So V satisfies the needed property. 

For all i G [d v — 1], let rij be the number of variable nodes u in G r for which the smallest integer / for which 
Equation (fT5T ) holds is / = i. Also, let no be the number of variable nodes u in G r for which there exists a 
path V of the form 

V : u c* v (16) 

where c* is a d c -regular check node of G r . Since in Equation (fT6l ) u has at most d„ neighbors in G r and 
c* is d c -regular, no < d v (d c — 1). Considering Equation (fT5T > with Z = 1, we note that has at most d v 
neighbors in G r and c\ is 2-regular. Thus, n\ < d v (d c — l)(d v — 1). Note that if u is a variable node in 
G r for which the smallest integer Z for which Equation (031 ) holds is Z = i + 1 (where i G [d„ — 2]), then 
there exists a path P that satisfies Equation (fT5l) with f i being a variable node in G r for which the smallest 
integer Z for which Equation (fl3T > holds is Z = z. Since for every I G [d v — 1] and every i G [Z], has at most 
d„ neighbors in G r and q is 2-regular, we have that rij + i < (d v — l)rij for all i G [cZ„ — 2]. By induction on 
i, we get that rtj < ^(dc — — 1)* for all i G [d„ — 1]. Thus, 



(d v - - 1 



d v -2 

i=0 i=0 

To show that To < qo, note that u G Wq if and only if there exists a directed path from u to f ma a; in G r 
containing only 2-regular check nodes. An analogous argument to the above implies that 



r <i+^(d,-ir 1 <i+ (4 ^ - = qo 



i=l 

□ 

Corollary 6.8. Let G r be the WDAG (with a single sink node) given in Lemma W5\ and z max be the maximum 



s ing 

regular check depth of a variable node in G r |lj Then, 



a m ax< max f(T ,...,T Zmax ) (17) 
(T ,...,T Zmax )eW 



where: 



°Note that in general z ma x < rmoi but the two quantities need not be equal. 
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and W is the set of all tuples (To, ■■■,T Zmax ) G ppm*ss+i satisfying the following three equations: 

Zmax 

^Ti = n r (18) 

To < q (19) 
For all i G {0, . . . , z max — 1}, T i+1 < qTi (20) 

where q = d v (d c — 1) ^"^ 1 j_ 2 ~* and qo = 1 + ^""^ _ 2 -• 

Proof of Corollary 16.81 The proof is similar to that of Corollary 15.91 Setting m = r max in Lemma 1631 and 
noting that the leaves of T have no entering flow, we get: 

E E ^ max>j ,k>F(u rmax )>(d c -i) r ^a max - e (^c-ir^EE 1 ^ 

i=i fc=i j=o j=i fc=i 

Thus, 

"max < E (d _ l)i E E r * J' fc 
i=0 V c ' j=l k=l 

Part [6] of Theorem 14. 1 1 implies that for every v G V r , the regular check depth of v in G r is equal to the 
minimum regular check depth in T of a replicate of v. By parts [3] and 0] of Theorem 14. II we also have that 

for all j G [n r ], E E Ti '^ k - 1 and for a11 * S {0, . . . , r mai .} and all k G [^j], r ijife < 1 and {Tij yk } iyk 

i=0 k=l 

all have the same sign. Thus, we get that: 



cw < E (d c -iy Tl 

where for every i G {0, . . . , r max }, Ti is the number of variable nodes with regular check depth equal to i 
in G r . Since T = for all z maa; < i < r max , we get that: 



a — < E vr^T) 



By the definitions of T and z max , T = w r . The facts that T + i < gTj for alii G {0, . . . , z max — 1} 



i=0 



and To < qo follow from Lemma 16771 □ 
Lemma 6.9. The RHS of rii7D is < c x nj~ e /or some constant c > depending only on d v and where 

Proof of Lemma I6.91 Let c = qp § 1 4 -. If n r > go> the claim follows from Theorem IA.6I with A = (/o> 

dc-l 

(3 = q and m = n r . If n r < qo, then the RHS of ( fTTl ) is at most n r < go < c, so the claim is also true. □ 

Proof of Theorem |6. 11 Theorem [67TI follows from Corollary 16781 and Lemma |6.9| by noting that \V r \ <\V\ 
since V r C V and that max \w(e)\ = 0( max c)|) by the hyperflow equation ©. □ 

eeE (v,c):w(v,c)<0 
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7 Relation between LP decoding on a graph cover code and on a derived 
spatially coupled code 

Definition 7.1. (Special variable nodes) 

Let C be a graph cover code and £' be a fixed element ofT>(Cf). Then, the "special variable nodes" of Q are 
all those variable nodes that appear in Q but not in Q'. 



Lemma 7.2. Let ^ be a (d v , d c = kd v ,L, M) graph cover code and let £' be a be a fixed element of T>(C)\ n \ 
Let n = (2L + 1)M be the block length of Q and consider transmission over the BSC. Assume a(n) is s.t, 
for any error pattern rj' on (', the existence of a dual witness for rf on (' implies the existence of a dual 
witness for rj 1 on (f with maximum edge weight < a(n). 

Then, for any error pattern rj' on £' and any extension rj of rf into an error pattern on the existence of a 
dual witness for rf on (f is equivalent to the existence of a dual witness for rj on Q with the special variable 
nodes having an "extra flow" of d v a(n) + 1. 

Proof of lemma 17.21 First, we prove the forward direction of the equivalence. Assume that there exists a 
dual witness for rf on ('. Then, there exists a dual witness for rf on £' and with maximum edge weight 
< a(n). This implies the existence of a dual witness for r\ on £ with the special variable nodes being source 
nodes and having an "extra flow" of d v a(n) + 1. 

The reverse direction follows from the fact that given a dual witness for 77 on (, we can get a dual witness 
for rf on £' by repeatedly removing the special variable nodes. The WD AG satisfies the LP constraints after 
each step since every check node in £' has degree > 2. □ 

Corollary 7.3. (Relation between LP decoding on a graph cover code and on a derived spatially coupled 
code) 

Let C be a (d v ,d c = kd v , L, M) graph cover code and let (f be a be a fixed element of T>((f). Let n = 
(2L + 1)M be the block length of Q and consider transmission over the BSC. Then, for any error pattern 
rj' on £' and any extension rj of rj' into an error pattern on £, the existence of a dual witness for rj' on £' 
is equivalent to the existence of a dual witness for rj on £ with the special variable nodes having an "extra 
flow" of d v cn 1 ~ t + lfor some c > and < e < 1 given in Theorem \6.1\ 

Proof of Corollary 17.31 By Theorem 16.11 the existence of a dual witness for rf on (' is equivalent to the 
existence of a dual witness for rf on (' and with maximum edge weight < cn 1_e for some c > 0. Plugging 
this expression in Lemma 17721 we get the statement of Corollary 17.31 □ 

8 Interplay between crossover probability and LP excess 

In this section, we show that if the probability of LP decoding success is large on some BSC, then if we 
slightly decrease the crossover probability of the BSC, we can find a dual witness with a non-negligible 
"gap" in the inequalities (0]) with high probability. 

Theorem 8.1. (Interplay between crossover probability and LP excess) 

Let C be a binary linear code with Tanner graph (V, C, E) where V = {v\, ■ ■ ■ ,v n }. Let e, 5 > and 
e' = e + (1 — e)S. Assume that e, e' , 5 < 1. Let q e i be the probability of LP decoding error on the e'-BSC. 

"Here, U(() refers to Definitionl23l 
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For every error pattern x G {0, l} n , if G = (V, C, E, w, 7) is a WDAG corresponding to a dual witness for 
x, let f(w) G M. n be defined by 

fi(w)= ^2 w(vi,c)- (-w{vi,c))= ^2 w ( v i> c ) ( 21 ) 

c£N(vi):w(vi,c)>0 ceN(vi):w(vi,c)<0 c€N(vi) 

for all i £ [n]. Then, 

Pr x ^Ber(e n)*P a dual witness wfor x s.t. fi(vu) < j(vi) — —, Vz G [n]} > 1 ^~ 

v ' ; 2 

In other words, if we let 7(^4) — fi(w) be the "LP excess" on variable node i, then the probability (over 
the e-BSC) that there exists a dual witness with LP excess at least 5/2 on all the variable nodes is at least 
1 _ 2 V 

Proof of TheoremED Decompose the e'-BSC into the bitwise OR of the e-BSC and the 5-BSC as follows. 

Let x ~ Ber(e, n), e" ~ Ber(5, n) and e = x V e". Hence, e ~ Ber(e', n). For every x G {0, l} n , we will 
construct a dual witness w x with excess 5/2 on all variable nodes by averaging and scaling the dual witnesses 

of xVe" where e" ~ Ber(5,n). More precisely, for every x G {0, l} n ,letw x = E e ii^Ber(5,n) {v xVe " } 
where v x is an arbitrary dual witness for x if x has one and v x is the zero vector otherwise. Note that w x 
always satisfies the check node constraints, i.e. for any x G {0, 1}™, any c G C and any v, v' G V, we have 
w x (v,c) + w x (v',c) > 0. We now show that, with probability at least 1 — over x ~ Ber(e,n), w x 
satisfies © with LP excess at least 5/2 on all variable nodes. For any weight function w : V x C — > M 
on the Tanner graph (1/, C, £% we define /(w) by Equation (T2TT) . For every x G {0, l} n , define the event 
L x = {x has a dual witness} and define x by Xi = (—l) Xt for all i G [n]. We have that: 

/K) = 7^S e ^B e r( 5 ,n){/(^ Ve ")} 
\1 2J 



2 

— jr{E e "^Ber{&,n){f{ wXS/e )\L xVe }Pr e n^Ber(8,n) {L XVe } 

V-*- ~~ 2' 

+ E e ^ Ber{s , n) {f(w xVe " ) \I^}Pr e ^ Ber{5 , n) {L^ 7 } 
(1 + 1) 



, h ^e»~ Be r(<5,n){/K Ve )l^ Ve }^ e ^ Ber((5iri) {^ Ve } (since £ e ^ Ber(M {/K Ve )|L*ve" } = ) 
(1 2) 

< ) _l E e "~Ber(6,n){x V e"|L xVe 'Vv'~Ber(,5,n){^ Ve ''} (by equation ©) 
(1 2/ 

= Si ~ h ( E e"~Ber(S,n){x V e"} - E e „^ Ber{Sn) {x V e"\L xS/e "} X ^ 
(1 2 ) 



where ^ = -Pr e //^ Ber ,( ( 5 jn ){L a:Ve "}. Note that for every i G [n], we have: 

E e "~Ber(5,n){x V e"} 



— 1 if = 1. 

+ (1 - 5){+l) = 1-25 if ^ = 0. 
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Moreover, E e ii^, Ber / Sn \\x V e"\L xye "} > — 1 since every coordinate of x V e" is > —1. Therefore, 

2 '-{-l + <p x ) i£ Xi = l. 



fi{w x ) < { 



(1 + 1) , 
(1-1) 
(1 + 1) 
1(1-1) 



(1-25 + 03.) i£xi = 0. 



We now find an upper bound on <f> x . Note that 4> x is a non-negative random variable with mean 



^B~Ber(e,n){0x} — E x ^ Ber ^{Pr e n^ Ber ^ n ^{L xVe "}} = Pr x ^Ber[e,n),e"~Ber{8,n) { L xVe " } 

= Pr e ~Ber(e',n){L e } = q e > (by TheoremE2]> 
By Markov's inequality, Pr x ^ Be r(e,n){<t>x > §} < Ik^r^lijfl = T h us , the probability over 

2 

x ~ 5er(e,n) that for all % G [n], /<(«>*) < |4y(-l + §) if x< = 1 and ftvf) < £±fj(l - f ) if 
Xi = 0, is at least 

5 S 2^ 

Prx~Ber(e,n){<l>x < £ } = 1 ~ Pr x~Ber(e,n){<t>x > ~} > 1 j~ 

Note that for all < <5 < 1, we have that 7^7(1 - ^) < 1 - §■ Thus, the probability over x ~ Ber(e, n) 
that fi{w x ) < (—l) Xt — I for all i € [n], is at least 1 — So we conclude that 

P r x~Ber(e n){^ a dual witness w for x s.t. < 7(i>i) — -, Vi G [n]} > 1 — — ^~ 

□ 



9 ice = £sc 

In this section, we use the results of Sections[6l|7]and[8]to prove the main result of the paper which is restated 
below. 

Theorem 9.1. (Main result: £,gc = £,sc) 

Let Tgc be a (d v , d c = kd v ,L, M) graph cover ensemble with d v an odd integer and M divisible by k. Let 
Tsc be the (d v ,d c = kd v ,L — d v ,M) spatially coupled ensemble which is sampled by choosing a graph 
cover code £ ~ Tgc an d returning a element ofT>(Q chosen uniformly at random^ 2 ] Denote by £gc an d 
£sc the respective LP threholds of Tgc an d Tsc on the BSC. There exists v > depending only on d v and 
d c s.t. if M = o{L u ) and T$c satisfies the property that for any constant A > 0, 

Pr c'~r sc [LP error on ('] = o(—^) (22) 

(H SC -A)-BSC L 

Then, £ G C = £sc- 

12 Here, 23(C) refers to Definitionl23l 
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Lemma 9.2. Assume that the ensemble T$c satisfies the property \22\ for every constant A > 0. Then, for 
all constants A\, A2, a, ft > 0, there exists a graph cover code £ G Yqq, with derived spatially coupled 
codes C'-l> ■ ■ ■ iC'l' satisfying the following two properties for sufficiently large L: 

1. Pr^ GC+ ^ 2 y BSC [LP decoding success on Q < a. 

2. For all i G [-L : L\, Pr^ sc _ Al y BSC [LP decoding error on < /3/(2L + 1). 

Proof of lemma [9^21 Note that a random code £ ~ ^gc satisfies the 2 properties above with high probabil- 
ity: 

Pr C~r GC [ p r{i GG +& 2 )-BSC [Success on C] > a or 3i G [-L : L] s.t. Pr^ sc _ Al y BSC [Error on > /3(2L + 1)] 

1 (2L + l) 2 
< —Pr c~r GC [LP decoding success on Q H Pr c~r sc [LP decoding error on £'] 

= o(l) 

Note that the inequality above follows from Markov's inequality and the union bound. We conclude that 
there exists a graph cover code £ G T^c satisfying the 2 properties above. □ 

Lemma 9.3. £ GC > £sc 

Proof of lemma 1931 We proceed by contradiction. Assume that ^gc < £sc- Let: 

5 = (£sc - Cgc)/2 
V = £sc - $ 

A = r, - 5/2 = i GC + 5/2 

Note that 77 > A + (1 — \)5/2. Let ( be one of the graph cover codes whose existence is guaranteed by 
Lemma l9!2l with Ai = 5, A 2 = 5/2 and a, [3 > with a < 1 - 2/3/(5 and let • • • , C' L be the spatially 
coupled codes that are derived from (. Let \i be an error pattern on £ and let ^ be the restriction of p, to 
for every i G [— L : L]. Define the event: 

E\ = {Vi G [-L : L], 3 a dual witness for /Xj on Q with excess (5/2 on all variable nodes} 

Then, 

Ei = [3i G [— L : L] s.t. $ a dual witness for \i{ on Q with excess (5/2 on all variable nodes} 

Thus, 

L 

P r \-BSc{Ei} < Pfx-Bsc{$ a dual witness for Q with excess 5/2 on all variable nodes} 

i=-L 

L 2 

< ^2 -^P r r]-BSc{^P decoding error on Q} (by Theorem 18 .It 

i=—L 

- 5 2L + 1 5 
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If event E\ is true, then by Corollary 17.31 for every I G [-L : L], there exists a dual witness {t\- \ i G V, j G 
C} for n on £ with the special variable nodes being at positions [1,1 + 2d v — 1] and having an "extra flow" 
of d v cn l ~ e + 1 with c > and e > given in Theorem 16. II and with the non-special variable nodes having 



excess |. Then, we can construct a dual witness for fx on the graph cover code ( (with no extra flows) by 
averaging the above 2L + 1 dual witnesses as follows. For every i G V and every j G C, let: 



1 L 
— T r l - 



avg 

Ti i ~ 2L + 1 ' lJ 



We claim that {r^- }ij forms a dual witness for \i on £. In fact, for each i G V, j G C and / G [— L : L], 
r\- + Tjij > which implies that: 



avg 



Moreover, for alH G V, we have that: 

1 L 

E avg = \- ( i- \^ 
^ V2L + 1 ^ 

jeN(i) jeN(i) i=-l 

L 



1 rf v t 1 



2L + 1 ^ \ ^ 13 

l=-L jeN(i) 

< ((d v ~ l)(d v c(M(2L + l)) 1 "' + 1 + 7i ) + (2L + 1 - (d v - l))( 7i - |)) 

(M(2L + 1)) 1 ^ , 4-1 ^ 

= 7i + (d„-lKc — + ^^ + ^^-2 

< 7 j if M = o(L u ), L sufficiently large and v = e/(l — e) 

Since Pr A . B5C {LP decoding success on C} > Pr\. B sc{Ei} = 1 - Pr\. B sc{Ei}, then, 

2/3 

Pr x.bsc {LP decoding success on £} > 1 — 

o 

which contradicts the fact that: 

2/3 

Prx-BSc[^P decoding success on (] = Pr^ GC+A2 y BSC [LP decoding success on (} < a < 1 — 

□ 

Lemma 9.4. £ GC < tsc 

Proof of Lemma 1941 Let ( be a graph cover code and D(() be the set of all derived spatially coupled codes 
of (. Let [i be an error pattern on ( and \J be the restriction of \i to C for some (' G -D(C)- Given a dual 
witness for /x on £, we can get a dual witness for // on £' by repeatedly removing the special variable nodes 
of (. Note that the dual witness is maintained after each step since every check node in £' has degree > 2. 
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So if there is LP decoding success for 77 on £, then for every £' G -D(C)> there is LP decoding success for 7/ 
on where 7/ is the restriction of r\ to Therefore, for every e > and every (' G D(Q, we have that: 

Pr e-BSC [LP decoding error on £'] < Pr e _Bsc[LP decoding error on (] 

This implies that for every e > 0, we have that: 

Pr ( i^ rs [LP decoding error on ('} < Prc~r GC [LP decoding error on (] 

i-BSC " e-BSC 

So we conclude that £qc < £,sc- D 
Proof of Theorem l9.11 Theorem [9Tl follows from Lemma [931 and Lemma |9~4l □ 



A Appendix 

A.l Proof of Theorem O 

The goal of this section is to prove Theorem 13 . 2 1 which is restated below. 
Theorem 13.21 (Existence of a dual witness and LP decoding success) 

Let T = (V, C, E) be a Tanner graph of a binary linear code with block length n and let r? G {0, 1}" be 
any error pattern. Then, there is LP decoding success for rj onT if and only if there is a dual witness for rj 
on T. 

Note that the "if" part of the statement was proved in [FMS+07]. The argument below establishes both 
directions. We first state some definitions and prove some facts from convex geometry that will be central 
to the proof of Theorem 13.21 

Definition A.l. Let S be a subset ofR n . The convex span of S is defined to be conv(S) = {ax + (1 — 
a)y I x,y G S and a G [0,1]}. The conic span of S is defined to be cone(S) = {ax + (3y \ x,y G 
5 and a, (3 G M>o}- The set S is said to be convex if S = conv(S) and S is said to be a cone if S = 
cone(S). Also, S is said to be a convex polyhedron if S = {x G R n \ Ax > b) for some matrix A G W nxn 
and some b G M n and S is said to be a polyhedral cone if S is both a convex polyhedron and a cone. The 
interior of S is denoted by int(S) and the closure of S is denoted by cl(S). 

Let K be a polyhedral cone of the form K = {x G W 1 \ Ax > 0} for some matrix A G W nxn . For any 
x G K s.t. i^O, the ray of K in the direction of x is defined to be the set R(x) = {Ax | A > 0}. A ray 
R(x) of K is said to be an extreme ray ofK if for any y,z G W 1 and any a, (3 > 0, R(x) = aR(y) + fiR{z) 
implies that y,z G R(x). 

Lemma A.2. If S is a convex subset ofW 1 , then int((R> ) n + S) = (M> ) n + S. 

Proof of LemmaEOl For all a G (M >0 ) n + S, a = r + s where r G (M>o) n and s £ S. Thus, the ball 
centered at a and of radius min ig [ n ] > is contained in ((M>o) n + S). Hence, a G int((M>o) n + S). 
Therefore, (M> ) n + SC int({R> ) n + S). 

Conversely, for all a G int((]R>o) n + S), a = r + s where r G (M>o) n and s G S. Moreover, since 
a G mi((R> ) n + S), there exists u G (M>o) n s.t. a + u G ((M> ) n + S) and a - u G ((M> ) n + S). 
Note that a + u = r + u + s and that a — u = r' + s' for some r' G (IR>o) n and s' G S. Thus, 

a = {a+u) + (a - u) = r _ ±2 ^ + s±^ = r „ + g n where r „ = r+^V G (R>o) „ and g li _ s^V g g sinc£ g 

is a convex set. Hence, int((R> ) n + S) C (R >0 ) n + S. 

Therefore, int((R> ) n + S) = {R >0 ) n + S. □ 
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Lemma A. 3. Let S±, ..,S P be finite subsets ofW 1 each containing the zero vector. Then, 



v v 
cone( conv(Sj)) = O cone(Sj). 

3=1 3=1 
P V 



Proof of Lemma |A.31 Clearly, cone( Q conv (Sj)) C p| cone(Sj). To prove the other direction, we first 

3=1 3=1 

P V 

note that G cone( O conv(Sj)). For any non-zero x G P| cone(Sj), we have that for all j G [p], 
x = a s,j s where for any s € 5j, a s j > 0. Let j m ax = argmax a s j. Since x ^ 0, D = 

X] as j™- > °- Thus ' for an y 3 e [p]. we have § = X] (^f ) s + ( : ~ X] "d") - Since for 

all j G [p], < a *,j — D and G S,-, we conclude that G conv(Sj) for all j G \p\. Hence, 

P P V 

x G cone( O conv(Sj)). Therefore, P| cone(Sj) C cone( P| conv(Sj)). □ 

Lemma A.4. Le? K be a polyhedral cone of the form K = {x G M m | > 0}/or iome matrix A G I^ xm 
of rank m. For any x G if s.t. x ^ 0, we have: 

1. If R(x) is an extreme ray of K, then there exists an (m — 1) X m submatrix A' of A s.t. the rows of A' 
are linearly independent and A'x = 0. 

2. K = cone(R) where R = \J R(x). 

extreme rays R(x) of K 

Proof of Lemma|A31 See Section 8.8 of HSch98L □ 

Lemma A.5. For all m > 2, we have that 

m 

{y G (R> ) m | y* ^ Mo ' Wi ° e H } = cone{z G {0, l} m | w(z) = 2} 

m 

Proof of Lemma POl Let K m = {y G (M>o) m | ^ y% > yj ,Vio G [m]} and X rn = cone{z G 

i=l, iy^io 

{0, l} m I w(z) = 2}. Clearly, X m C if m . We now prove that K m C X m . Note that K m can be written in 
the following form: 

m 

lf m = {y G M m | yi > Vi G [m] and ^ 2/i > j/ io , Vi G [m]} 

i=l, i^irj 

= {t/£M m |ij/>0} where 4 G M 2mxm has rank m 
By part l2l of Lemma lA!4l we then have: K m = cone(R) where R = [j R(y)- Therefore, 

extreme rays R(y) of K m 

by part Q] of Lemma lA!4l it is sufficient to show that if y G M m satisfies any (m — 1) equations of ET m with 
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equality, then y should be an element of cone{z G {0, l} m | w(z) = 2}. Note that we have two types of 
equations: 

m 

(i) ^2 Vi ~ yi ° = for some io G H- 

i=l, i^i 

(II) Hi = for some i G [m] . 

Consider any (m — 1) equations of if m , satisfied with equality. We distinguish two cases: 
Case 1: At least (m — 2) of those equations are of Type (II). Without loss of generality, we can assume that 
Hi = for all i € {3, ... , m}. Moreover, since y G K m , we have that y\ — yi > and yi — y\ > 0, which 
implies that y\ = yi. Therefore, we conclude that y = yx(l 1 ... 0) T G X m . 

Case 2: At most (m — 3) equations are of Type (II). Hence, at least 2 equations are of Type (I). Without loss 

m m 

of generality, we can assume that yi = y\ and Vi = V2- Adding up the last 2 equations, we 

1=1, i^l j=l, 

m 

get yj = 0. Since y G K m , we have yi > for alii G {3, . . . , m}. Therefore, we get y« = for all 

i=3 

i G {3, . . . , m}. Similarily to Case 1 above, this implies that y G -X" TO . □ 

Proof of Theorem 13.21 The "fundamental polytope" P considered by the LP decoder was introduced by 

HKV031 and is defined by P = f] conv(Cj) where Cj = {z G {0, l} n : w{z\N(j)) is even} for any 

jec 

j G C. For any error pattern rj G {0, l} n , let rj G {—1, l} n be given by rji = (— 1)* for all i G [n]. Also, 

n 

for any x, y G M n , let their inner product be (x, y) = &iyi- Then, under the all zeros assumption, there 

i=i 

is LP decoding success for rj on £ if and only if the zero vector is the unique optimal solution to the LP (O, 
i.e. if and only if (rj, 0) < (rj, y) for every non-zero y G P, which is equivalent to rj G int(P*) = intQC*) 
where K = cone{P} is the "fundamental cone" and for any S C IR n , the dual S* of S is given by S* = 
{z G M n | (z, x) > \/x G S}. By Lemmas |A3] and |A3J we have: 

/C = cone(^conv(Cj)) = ^cone(Cj) = ^cone{z G {0, l} n | w(z\nq)) is even} 
jec jec jec 

= p| cone{z G {0, 1}" I w(z\ m ) = 2} = f| {y G (M>o) n I ^2 Vi > yio' Vi o G A r (j)} 
jeC jec ieN(j)\{i } 

= {y G (M> ) n | > Vi G N(j), Vj G C} 

where Vi j G {—1,0, l} n is defined as follows: For all % G [n], 

'0 ifi£N(j). 
— 1 if i = io- 

1 ifi e JVO*)\{*o}- 



Thus, 



K = (M>o) n n Pi (coneKjIio G ^0')})* = (^o)^ f| 

where for any j G C, Dj = cone{vi j\io G iV(j')}. Note that if L C M n is a cone, then its dual L* is also 
a cone. We will use below the following basic properties of dual cones: 
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i) If Li, L 2 C M n are cones, then (L x + L 2 )* = £* n L\. 

ii) If L C M n is a cone, then (L*)* = d(L). 

Therefore, there is LP decoding success for rj on /C if and only if 77 € £> where: 



D = int(K,*) = mt(((R> ) n f) D j)*) = «i*((((K>o) n )* f| fl D ^Y) = int ((((^>oT + Y, D i 



j&C ' v ' jgC 7 v j&C 

and where the third equality follows from the fact that (M>o) n is a self-dual cone and the last equality 
follows from property (i) above. Note that for any j G C, Dj is a cone. Moreover, since (M>o) n is a cone 
and the sum of any two cones is also a cone, it follows that (R>o) n + Dj is also a cone. Furthermore, 

by property (ii) above, we get that D = int ( cl ^(M>o) n + Dj^j J . Being a cone, (M> ) n + ^ 

Dj is a 

convex set. For any convex set S C M n , we have that int(cl(S)) = int(S) (See Lemma 5.28 of [AB06]). 
Therefore, 

D = int((R> ) n + ^2Dj) 

3&C 

= (IR>o) n + Dj (using Lemma |A!21 and the fact that Dj is a convex subset of M n ) 

= {z G R n I 3y G ^ ^ s.t. z > y} 

= {z G R n I 3{A io j} ioe7V (j)j e c s.t. A i0ii > Vi G iV(j'), Vj G C and ^ KojV iQtj < z) 

ioeN(j),jec 

= { Yl X ioJ v io,j + u I K,j > Vi G iV(j), Vj G C and tt G (M >0 ) n } 
io£N(j),j£C 

Thus, there is LP decoding success for 77 on ( if and only if there exist Aj 0j j > for all i G iV(j) and all 

j EC s.t. ^ Kj v k,,j < V- Let j) = ( ^2 A - •/'''• ,/), for a11 * 6 [n] and all j G C. Since 

ioeN(j),jeC ioeN(j) 
( v io,j)i = whenever i £ N(j), we have that for every i G [n]: 

u>(«,j) = ( Y X io,j v io,j)i = J2( Y X k>,j v io,j)i= { Y X ioJ v io,j)i<m 

jeN(i) jeN(i) i eN(j) j<=c i eN(j) ioeN(j),jeC 

Moreover, for all j G C, i\, i<i G -/V(j) s.t. ii 7^ 22, we have 

w(tl,i) + = Yl X ioj{{ v io,j) h + ( u <oj)ia) ^ 

i eW(i) 

since (w^j)^ + (^i ,i)j > because ii 7^ «2 G iV(j). We conclude that LP decoding success for 77 on £ 
is equivalent to the existence of a dual witness for 77 on (. □ 
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A.2 Proof of Lemmas EZEQ] and M\ 

The goal of this section is prove the following theorem which is used in the proofs of Lemmas [5.101 and |6T9l 

Theorem A.6. Let A, /3, m be positive integers with j3 > d c — 1 and m > X. Consider the optimization 
problem: 

v* = max /(T , . . . , T h ) (23) 

(T ,...,T h )ew h 

h£N,h>l 

where: 

" Ta 



/(T ,...,T,)-^— 



i=0 



and Wh is the set of all tuples (To, . . . , Th) G N fe+1 satisfying the following three equations: 

h 

^Ti = m (24) 

i=0 

T < A (25) 
T l+ i < pTifor all i G {0, . . . , h - 1} (26) 

* <r \ fcl) 1 ^-! n( fl dc - 1) 

f < A — ■£ m ^ 

_2 i 

d c -i 1 

We will first prove some lemmas which will lead to Lemma [A3] 
Definition A.7. Let I = [log^ "^ 1 ) + 1)J - 1. 
Note that I > since m > A. 

Lemma A.8. Let (T , . . . , T h ) G W h . Then, T< < X^ for all i G {0, . . . , /i}. 

Proof of Lemma |A78l Follows from equations (1231 ) and (|26l ). □ 
Lemma A.9. Let 

T[ = Xfi for alii e {0,...,/} 
(g+j - 1) 

Then, m,...,T{ +1 )eW l+1 . 

Proof of Lemma[OJ First, note that (Tq, . . . , T/ +1 ) G N /+2 since T/ +1 > by Definition [O Moreover, 

i=0 i=0 { ^ ' 
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We have that Tq < A and for every i G {0, — 1}, T/ +1 < /3T/. We still need to show that T/ +1 < /3T/. 
We proceed by contradiction. Assume that T/ +1 > (3T[. Then, T/ +1 > X(3 l+1 . Thus, 



'+ 1 '+ 1 i\ / m(/3-l) -.x "| 

* E ^ = > (^1) ^ '" 



i=0 i=0 

since / + 2 = [log^^^ + 1)J + 1 > lo g/3 (^^ + 1). □ 

Lemma A. 10. (Tq, . . . ,T{ +1 ) is the unique (up to leading zeros) element that achieves the maximum in 
Equation (12 3i . 

Proof of Lemma lAlOl By Lemma[A9l (T^, . . . , T/ +1 ) G W i+1 . Let (T , . . . ,T h ) G Ty h such that (T , . . . ,T ft ) 
and (Tq, . . . , T' h ) are not equal up to leading zeros and without loss of generality assume that h > I + 1 by 
extending T with zeros if needed. In order to show that /(To, . . . , Th) < /(Tq, . . . , T' h ), we distinguish two 
cases: 

Case 1: (T , . . . , Tj) / (T^, . . . , T/). By Lemma|AH there exists k\ G {0, ...,/} such that T fel < Xj3 kl . 
i i 

Therefore, ^ T- - V" Tj > 0. Note that: 

i=0 i=0 

T 1 T' T/ ^ T' r7-T 

/(T , ...,T h )- /(T^, . . . , T/ +1 ) = 1 _ 1 + _ lV +i + J] 



< t VfT - T') + +1 - 1+1 + - V T 

- (d c - i)' ^ + (d c - i)«+i + (d c - i ~j^_ 2 1 

i 1 l h 



Consequently, 



(E^-E 1 *) (E^-E r <: 
/(r ,...,r h )</(^,...,7r + i)- i= ° + ' " 



(dc - 1)' (d c - 



= /(T-,...,T^)-(d c -2) ^npi 
</(T^,...,T/ +1 ) 



7i 

Case 2: (T , . . . , T { ) = (Tq\ . . . , T/). Then, T l+1 ^ T/ +1 . Since T/ +1 = ^ T, we should have T/ +1 - 

i=i+i 
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Ti + i > 0. We have that 



rj~i rj~if h 

/(T , . . . , T h ) - f(%, . . . , T/ +1 ) = — - *+* + 

. Ti+i - T( +1 



+ (A _ 1 V+2 Ti 



Consequently, 



Tf+i - t/ +1 (r/ +1 -r m ) 



j{J-0,---,lh) < /Uo' • • • > J /+lJ 7^ n m + 



(d c - iy +i (d c - iy+ 2 

= f(T>,...,Tl +1 )-(d c -2) { ^-^ 
<f(T^...,T[ +1 ) 



Proof of Lemma lAT6l Let v = /3/(d c — 1). By Lemmas [A.10I and fA. 81 we have that 

v *<y _Ii < y x—l— = aW = a^^I < a— 



< a " — < A-^z^" 1 

v — 1 z/ — 1 

J/ 2 w 

< A m lYi f 

~ u-l 

□ 

A.3 Proof of Theorem I57TT1 

The goal of this section is to prove Theorem 15 . 1 1 1 which is restated below. 

Theorem |5.11l (Asymptotic tightness of Theorem \5. 1 1 f or (d v , d c )-regular LDPC codes) 
There exists an infinite family of (d v , d c )-regular Tanner graphs {(V n , C n , E n )} n , an infinite family of error 
patterns {jn}n and a positive constant c s.t. there exists a hyperflow for 7 n on (V n , C n ,E n ) and any WDAG 
(V n , C n ,E n ,w, 7 n ) corresponding to a hyperflow for 7„ on (V n , C n , E n ) must have 

max|u;(e)| > cn ln ( d "- 1 )+ ln ( dc - 1 ) 



2 



□ 



We now prove some lemmas that lead to the proof of Theorem 15.1 II 
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Definition A.ll. (Construction of {(V n , C n , E n )} n ) 

Let f3 = (d v — l)(d c — 1). The Tanner graph {(V n , C n , E n )} n is constructed by connecting copies of the 
following two basic blocks: 

1. The "A block" A x with parameter the non-negative integer x. A x is an undirected complete tree rooted 
at a (d v — l)-regular variable node. The internal nodes of A x other than the root are either d c -regular 
check nodes or d v -regular variable nodes. The leaves of A x are all 1-regular variable nodes of depth i@ 
Thus, A x has (3 X leaves. An example A block is given in Figure\2\ 

2. The "B block" B y with parameter the non-negative integer y. B y is an undirected tree rooted at a (d v — l)- 
regular variable node. The internal nodes of B y other than the root are either d v -regular variable nodes 
or 2-regular check nodes. The leaves of B y are 1-regular variable nodes. The nodes of B y are divided 
into y + 1 layers indexed from y to 0. Layer y consists of the root and the [d v — 1) check nodes that are 
connected to the root. Each check node in layer i is connected to a single variable node in layer i — 1 for 
all i = y, y — 1, . . . , 1. Each variable node in layer i is connected to d v — 1 check nodes in the same layer 
for all i = y, y — 1, . . . , 1. Thus, layer consists of (d v — l) v leaves which are all 1-regular variable 
nodes. An example B block is given in Figure\3\ 

Let 7 = i n ( dv l ^+^ dc _iy For every non-negative integer n, let y n = Llog^-i) n 7 J andb n = (d v — l) Vn = 
G(n 7 ). The Tanner graph {(V n , C n , E n )} n is constructed using a root check node, one B block, many A 
blocks and some auxiliary variable and check nodes as follows: 

1. Start with a check node cq. 

2. Connect cq to the roots of d c — 1 A yn+ i blocks and to the root of one B yn block. Note that B yn has b n 
leaves. 

3. For every i = y n , y n _i , . . . , 1, connect each check node in layer i of B yn to the roots of(d c — 2) Ai blocks. 
Note that there are (d v — l)^™ - ^ 1 check nodes in layer i. 

4. Let T n be the tree constructed so far and l n be its number of leaves. Note that all the leaves of T n are 
1-regular variable nodes. Complete T n into a (d v ,d c ) -regular graph by adding 0(l n ) d c -regular new 
check nodes and (if needed) 0(l n ) d v -regular new variable nodes in such a way that each new check is 
either connected to zero or to at least two leaves oftheBblock^ 

We call the check and variable nodes added in step @ the "connecting " check and variable nodes respec- 
tively. 

Definition A.12. (Construction of {^ n } n ) 

Let {{V n , C n , E n )} n be the Tanner graph given in Definition \A. 1 1\ The error pattern j n is defined by: 

1. For every variable node v in an A block, 7 n (w ) = 1. 

2. For every variable node v in the B block, 7 n (w) = — 1- 

3. For every connecting variable node v, 7 n (w) = 1- 

13 The depth of a variable node v is the number of check nodes on the unique path from the root to v. 

14 Note that if (d v — l)l n is divisible by d c , we don't need any extra variable nodes. In the worst case, we can add d c copies of 
T n so that (d v — l)d c l n is divisible by d c . 
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Figure 2: Example of an A block with parameter x = 1 where d v = 3 and d c = 4 
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Figure 3: Example of a B block with parameter y = 2 where d v = 3 



Lemma A. 13. (Size of the code) 

For any positive integer n, the Tanner graph {(V n , C n , E n )} n given in Definition \A. 1 1 1 is a (d v , d c )-regular 
code with Q(n) variable nodes. 

Proof of Lemma |A.131 It is enough to show that the number l n of leaves of T n is 0(n). The number of 
leaves of block B yn is b n = ©(n 7 ). The number of leaves of block A y is (d v — l) y . Thus, the number of 
leaves in all the A-blocks is 

Vn 

a n = (d c - l)(d v - + (d c - 2) J2(d v - l)y-- l+l p i 

i=i 

= o((d v - i)»o + o((d v - i)y- Y,( d c - 1)') 

i=l 

= o{b n + py-) 

Vn 

because (d v - = b n and ^(d v - Vf = 0((d c - l) Vn ). Since /3 Vn = G(n) and b n = o(n), we get that 

i=i 

l n = K + a n = 6(n). □ 
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Lemma A.14. (Existence of a hyperflow for {"f n }n on {(V n , C n , E n )} n ) 

Let {(V n , C n , E n )} n be the Tanner graph given in Definition \A.11\ and let 7 n be the error pattern given in 
Definition \A.12\ Then, for every positive integer n, there exists a hyperflow for 'jn on (V n ,C n , E n ). 

Proof of Lemma lA.141 Let e > 0. We will further specify e at the end of the proof. Consider the following 
assignment of weigths to edges of E n : 

1. In every A block, the edges are directed toward the root of the block. The edges outgoing from the leaves 
have weight 1 — e. For every check node, the weight of the outgoing edge is equal to the common weight 
of its incoming edges. For each variable node, the sum of the weights of the outgoing edges is equal to 
the sum of the weights of the incoming edges plus 1 — e. Thus, the weight of the edge outgoing from the 
root of the A x block is 

r x = (1 - e) f> - If = (1 - e) ^" 1 ^ 1 " 1 

2. In the B block, the edges are directed toward the leaves. The edge connecting cq to the root of block B 
has weight w Vn where for any i € {0, . . . ,y n }: 



(i + e) j^{d v - iy = (i + £ ) (d » -ig 1 



Wi 

3=0 

For every internal variable node v, the weight of each outgoing edge from v is ^r~~i wnere z is the 
weight of the edge incoming to v. For every internal check node c, the weight of the edge outgoing from 
c is equal to the weight of the edge incoming to c. By induction on the layer index i = y n: y n -\, . . . , 0, 
for every variable node v in layer i, the weight of its incoming edge is Wi and (if v is not a leaf) the 
weight of each of its outgoing edges is Wi-i (since wi satisfies the recurrence Wi-i = Wl, ^]X^ f° r an 
i = Vn,Vn-l, •••,!)■ 

3. All edges adjacent to connecting check or variable nodes have weight zero. 

By construction, the weights satisfy the dual witness equations (01) and (f5]) for all check and variable nodes 
in A blocks, all internal variable nodes in the B block and all the connecting check and variable nodes. To 
guarantee that equations ((U) and (f5]) hold for the root check node cq, we need that r Vn+1 > w Vn . To guarantee 
them for the internal check nodes of the B block, we need that r%+\ > Wi for all i = y n — 1, . . . , 1. To 
guarantee them for the leaves of the B block, we need that wq — 1 > 0, which holds since wq = 1 + e. Thus, 
for every i = y n , y n -i, . . . , 1, we need that r^i > tOj, i.e. 

which can be guaranteed by letting < e < 1 — j-. □ 
Lemma A.15. (Lower bound for any hyperflow for {~f n }n on {(V n , C n ,E n )} n ) 

For any positive integer n, any WDAG (V n , C n ,E n ,w, j n ) corresponding to a hyperflow for 7 n on (V n , C n , E n ) 
must have 

ln(d„-l) 

max|w(e)| > cnM^-iJ+MJc-i) 

for some constant c > 0. 
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Proof of Lemma IA.151 Let ( V n , C n , E n , w , 7 n ) be a WD AG corresponding to a hyperfl ow for 7„ on (V n , C„ , E n ) . 
Since 7 n (i>) = —1 for every leaf v of the B block (which has b n leaves) and since each connecting check 
node adjacent to a leaf of the B block is connected to at least two leaves of the B block, there should 
be a flow of total value larger than b n from the non-leaf and non-connecting nodes of the B block to its 
leaves. Applying the same argument inductively and using the fact that for every variable node v of the B 
block 7n(t>) = —1, we get that all the edges of the B block should be oriented toward its leaves and that 
there should be a flow of value larger than b n entering the root of the B block. Thus, the edge connecting 
Co to the root of the B block should be oriented toward the B block and should have value larger than 

ln(d u -l) 

6 n = e( n ln(d„-l) + ln(d c -l))_ □ 

Proof of Theorem I5XT1 Follows from Lemmas lATT3llA~T4l and |A~T51 □ 
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